Large databases of TV advertising commercials are currently maintained by many types of entities which use such databases for a myriad of purposes. Many of these databases are built by identifying TV advertising commercials in video data streams that are “aired” during a broadcast, or “streamed” during an internet-based viewing session. The video data stream typically does not contain any information regarding the product or brand that is advertised in the commercial, so these entities must extract such information from the audio and/or video of the commercial. Once extracted, this information is tagged as metadata and is stored in a database of “known commercials” that is used for a myriad of functions.
In one known process, a content processing platform uses any combination of automated, semi-automated and manual (human-aided) processes to determine if a particular video segment that has been identified as a potential commercial is actually a commercial. If so, the content processing platform tags the commercial with appropriate metadata and adds the commercial and its metadata to a library of known commercials. FIG. 3 of U.S. Pat. No. 9,628,836 (Kalampoukas et al.), which is incorporated by reference herein, shows an example of how commercials are stored with identification information and metadata. The metadata may include information such as the product and/or brand for the commercial (e.g., Toyota®, Camry®). Referring again to FIG. 3 of this patent, some of the metadata may also be used for the title of the advertisement. That is, an advertisement for a Toyota Camry may be tagged with the metadata of “Toyota” and “Camry” but is also titled “Toyota Camry Commercial.” The title may be selected by a human or may be machine-generated.
Conventional metadata tagging involves a human watching the commercial and manually inputting the metadata. However, automated and semi-automated processes now exist for performing metadata identification, with human (manual) interaction being either eliminated or limited to spot-checking the automated process. Despite the new automated processes that have been recently introduced, there is still a need to improve the accuracy of the metadata identification process. The present invention fulfills such a need.
Prior art processes exist to analyze image frames and identify objects in the image frame. For example, the GOOGLE® Cloud Vision API provides image analytics capabilities that allows applications to see and understand the content within the images. The service enables customers to detect a broad set of entities within an image from everyday objects (e.g., “sailboat”, “lion”, “Eiffel Tower”) to faces and product logos. However, simply knowing that an object or product logo is present in an image frame does not provide a sufficient degree of certainty that such information can be used for metadata tagging of a commercial, particularly, for “primary metadata.” As defined herein, “primary metadata” is metadata regarding the commercial that directly identifies the product and/or brand being advertised in the commercial. As also defined herein, “secondary metadata” is any metadata regarding the commercial, other than the primary metadata. Secondary metadata is not necessarily stored in a database of commercials. However, depending upon how the database is used, it may be useful to store selected secondary metadata. Secondary metadata may include entities identified in the commercial which are not the product or the brand (e.g., roadway, sign, water), or it may be a product type or product category associated with the product or brand (e.g., vehicle, SUV).
Consider a Toyota Camry commercial that shows the vehicle driving past sailboats and the Eiffel Tower. Conventional image analytics such as the GOOGLE Cloud Vision API might identify the following five entities: sailboat, Toyota logo, Camry, Eiffel Tower and roadway. The Toyota logo and “Camry” may be identified using image comparison, logo detection, or optical character recognition (OCR). Products such as Microsoft® Azure Media Analytics can perform video OCR.
Conventional image analytics may also identify the Toyota Camry as simply being a “vehicle” or a “car.” However, the conventional image analytics is not designed to identify the primary metadata for such a commercial, which is only “Toyota” and “Camry.” The other identified entities, are, at best, secondary metadata, as defined herein. That is, “vehicle” or “SUV” is not the product and/or brand being advertised in the commercial, but could represent any one of a plurality of products and/or brands. Thus, there is still an unmet need to identify primary metadata among all of the detected metadata. The present invention fulfills such a need.